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1. Introduction 



Even in an emergency situation, it is difficult to speed up the 
production rate of sophisticated, modern weapons. The time constant for 
increasing production rate for many weapons seems to be on the order of a 
year, whereas major wars are sometimes imagined to last for only several 
months. For many weapons, in fact, it is not a bad approximation to assume 
that a major war would have to be fought entirely with war reserve stocks. It 
might be possible to accomplish a certain amount of geographic redistribution 
during a war, but not much production. 

Given these supposed facts, the following question would seem to be 
crucial for the yearly POM process: "How should a fixed budget be spent 

augmenting the current stockpile of weapons so as to maximize the 
effectiveness of the resulting stockpile?" Operations Research technioues 
could play an important role in answering the question, since several 
favorable preconditions exist: 

* The question must be asked repetitively. 

* Combat modelling must inevitably be involved in assessing 
effectiveness. 

* Lots of data are available that must be taken into account. 

* The problem of determining the best stockpile can be interpreted as 
one of mathematical optimization. 

In the Spring of 1984, a "Stockpile seminar" was held at HPS, the object 
of which was to review whether the crucial question mentioned above actually 
gets asked, and also to determine the role of Operations Research in answerinn 
it. The present document is a record of the deliberations and observations 
made during and after that seminar. 
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The seminar was run and attended mainly by faculty in the Operations 
Research Department, although one of us (Boger) is in the Administrative 
Sciences Department, and some OR PhD students also participated. The goal of 
the seminar was to teach each other about and criticize the methods and models 
used by the several armed services, both for tutorial reasons (this report 
will be used as a handout) and because some of us are interested in doing 
further research in what appears to us to be a vital area for the Department 
of Defense. The seminar was held at the one-hour-per-week level throughout 
most of the Spring quarter of 1984, climaxed by one all -day session attended 
by several outside experts. A list of attendees at that session is included 
as Appendix 1. 

We are not the first to conduct a comparative analysis of methods for 
determining stockpiles. References [1,8,10] are examples of previous studies. 
The present report is flawed, moreover, in not being comprehensive; the topic 
is a large one, and there are many areas that we simply didn't look into. In 
particular, we v/ere unable to find out very much about US Army methods. 
Nonetheless, v/e will make some observations that are not contained in previous 
studi es. 

Exceot for the Navy's use of the 0PM model (see sec. 2.5), the crucial 
question above doesn't get asked. Instead, the question is "What stockpile is 
needed to satisfy our requirements," or sometimes "What is the cheapest 
stockpile that will satisfy our requirements?" The typical answer to this 
question is that, even when costs are minimized, the required stockpile is 
considerably more expensive than is fiscally feasible. Figure 1, for example, 
shows for a typical weapon the comparison between inventory and the Navy's 
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"proqrarmi nq objective profile" as determined by the NMOR ( s’ec 2.2). There is 
clearly a large difference between the two, particularly if the gap is 
compared to the yearly stockpile increment. One way of resolving the 
discrepancy between budgets and requirements would be to reassess requirements 
(possibly also budgets) until feasibility is finally achieved. That is not 
typically done, however. We are not sure exactly what is done, but the models 
used to determine the requirements-driven stockpiles in the first place 
apparently play only a minor role. 
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There are several reasons, some of them good ones, why the armed services 
persist in asking what appears to be the wrong question. Our feeling is that 
the system (the national defense of the USA) would be better off if budget 
constrained questions were asked in the first place, but we will spare 
the reader our standard academic rant on the subject. Suffice it to say 
that if weapons cannot compete against each other for representation in the 
stockpile, then it is going to be difficult to make the cost/effectiveness 
tradeoffs that are essential in developing a proper weapons mix. These 
comments are more related to the budgeting system than to any of the 
quantitative models used therein, but there are additional weapons mix 
biases built into the models themselves, as will be seen. 

2 . Current Navy Methods 

The methodologies that are presently being used by the Navy to generate 
ordnance requirements for the POM are known collectively as the Non-Nuclear 
Ordnance Requirements (NNOR). The NNOR is composed of two separate approaches 
to the modelling of combat requirements for major ordnance exoendables. The 
first methodology addresses only the air-to-surface requirements. It is a 
"level of effort" method, by which is meant that the need for weapons is 
limited by the number of shooters, rather than the number of targets. The 
second is a "threat-oriented" methodology wherein the need for weapons is 
determined mainly by the number of targets. The NNOR generates requirements 
for all other major ordnance expendables. 

2.1 Level -of-Effort Methods (NAVMOR) 

Air-to-surface requirements within the NNOR are calculated via three 
computer models; PHASER, NAVMOR, and MRG. PHASER is the model which generates 
the numbers of sorties flown and aircraft attrited in various scenarios across 
several theaters. These outputs from PHASER become inputs for NAVMOR and MRG. 
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PHASER accepts as inputs such items as carrier employment schedules, 
aircraft inventories and locations, sortie rates, attrition rates, weather 
factors, and maintenance factors. A separate PHASER run is required for each 
aircraft type across all scenarios, theaters, and time. PHASER outouts are 
generated within fixed time intervals (phases) during which sortie input 
parameters remain fixed. Sortie rates and attrition rates are adjusted for 
weather and maintenance effects. PHASER keeps track of the expected aircraft 
assets during each phase of a scenario, but not the distribution of assets. 

The output of sorties flown may be decomposed by mission types. Finally, 
these output estimates are divided into general -purpose weapon sorties and 
special-purpose weapon sorties. This percentage allocation is made on the 
basis of subjective judgments of experienced military personnel. The 
general -purpose weapon sorties are used as inputs to HAVMOR, and the 
special-purpose sorties are inputs to MRG. 

MAVMOR (Navy and Marine Ordnance Requirements) is the computer model used 
to determine combat requirements for ordnance expendables using 
PHASER-generated attack sorties for general -purpose weapons. The general 
methodology used in NAVMOR is that total sorties input to the model are 
allocated over different prespecified sortie types. A cost competition 
determines the optimal MAVMOR weapon type for each sortie type defined to be 
armed with a MAVMOR weapon. 

This cost competition is really the heart of MAVMOR, so it is worthwhile 
looking at it in some detail. The competition takes the form of maximizing 
the expected number of kills per dollar by selecting optimally the decision 
variables (w,i,t,n), where i is the number of weapons of type w loaded onto a 
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sortie that will make n passes using tactic t. The expected number of target 
kills, k(w,i,t,n), is determined from the Joint Munitions Effectiveness Manual 
(JMEM), and the expected number of kills per dollar is given by 



(i*U(w))+0+(R*P^(w,t,n)) 

where 

U(w) = cost per weapon type w, 

0 = operations and maintenance cost per sortie, 

R = aircraft and crew cost, and 
P|^(w,t,n) = one pass attrition probability. 

Additionally, P|^(w,t,n) is given by 

Pj,(w,t,n) = 1 - (1 - xS(w,t))", (2) 

which the reader will recognize as the formula which applies when a fixed 
number, n, of passes is made with an attrition probability oer pass of 
XS(w,t). The function S(w,t) comes from relative, judgmental inputs of the 
form "attrition using tactic t is (say) double that of standard tactics." The 
attrition scaling factor, X, is designed to convert such relative judgments 
into one-pass attrition probabilities. Formulas (1) and (2) are essentially 
formula (2-23) of reference [6], except that additional parameters determining 
target type, aircraft type, mission type, weather, and time of day have been 
suppressed. 

After determining (w,i,t,n) in each case based on the cost/effectiveness 
of formula (1), total attrition can be obtained by summing over the additional 
parameters mentioned above. This total attrition is forced to meet a pre- 
determined constraint by adjusting the attrition scaling factor x until the 
calculated attrition is judged to be realistic. This procedure permits expert 
attrition judgments to be made on a relative scale while preserving an 
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absolute attrition constraint. This desirable feature is achieved at some 
cost, however, as will be discussed further in section 2.3. 

The Miscellaneous Requirements Generator (MRG) is the computer model used 
to calculate all level-of-effort combat ordnance requirements not generated by 
NAVMOR. MRG calculates requirements on an ordnance item basis with no 
competition between items for the arming of sorties. For each item, the 
special-purpose sortie output from PHASER is used to find the sortie estimates 
for any aircraft type compatible with the item. These sorties are then 
allocated over mission and operation type based uoon subjective military 
judgment. The basic computational concept of MRG is that combat requirements 
are equal to the product of total sorties flown, the fraction of those sorties 
on which the item is used, and the number of items expended per utilization 
sortie. Accumulation of these requirements over all sortie types yields total 
combat requirements for that item. 

2.2 Threat-Oriented Methods (the THREAT model) 

The Havy uses a static methodology to calculate combat reouirenents for 
threat-oriented environments. These environments include such missions 
as AAW, ASUW, and ASW. By changing certain elements of the underlying 
methods, the same methodology can apply to all three types of missions. 

The threat-oriented methodology begins by partitioning the threat; that 
is, it separately treats each ordnance item for which combat requirements are 
to be generated. The methodology uses a generalized "shoot-look-shoot" oolicy 
in which independent salvos, not single shots, are fired sequentially at 
sequentially arriving targets. 

This policy results in a number of required salvos that is a geometric 
random variable with a mean equal to the reciprocal of the salvo kill 
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probability. It was presumably this feature that the Secretary of the Navy 
had in mind when he said: 

"It took us the first full year of the Administration to turn 
around the totally unrealistic peacetime planning models that the 
analytical community had foisted on the operators. You could only 
buy two torpedoes for every target in the Soviet fleet that was 
worth a torpedo, because you had say a 55% or 65% kill probability, 
and so two gave you over 100% and, therefore, you could not buy any 
more. That's the situation we were in; it was totally unrealistic." 

-John Lehman, Armed Forces Journal, 1983 

Note that it is the low performance weapons that are emphasized in the 
threat-oriented method, since many weapons are required to do the job. This 
tendency to use many weapons is to some extent mitigated by truncating the 
geometric distribution to account for the inability of platforms to fire at 
particular targets, such as fast-movers, until they obtain a kill. All 
probability remaining at salvo values larger than the maximum depth-of-f ire 
value is lumped with that at the maximum depth-of-f i re. 

One problem that the Navy must face in stockpiling weapons is the 
maldistribution of targets over platforms. The obvious analytic assumption 
that each target is independently equally likely to encounter each platform is 
unduly optimistic, since historical experience is that targets are much more 
concentrated than that. The assumption actually made is that target 
encounters follow a "contagion" model wherein the probability that each 
target encounters a given platform is proportional to the number of targets 
already encountered by that platform. The parameter p is each platform's 
"initial experience," with P = 0 corresponding to a situation where some 
unlucky platform encounters every target, and p = « corresponding to the 
earlier rejected assumption of perpetual equal likelihood. Although there is 
no evidence that this contagion model provides an especially good fit to 
history, it is no less plausible than other models, and there is enough 
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historical evidence to establish that the best value for p is approximately 
1.0. When there are many targets and platforms, p = 1 corresponds to a 
geometric distribution for the number of targets encountered by a platform, 
whereas the case p = ® would correspond to a Poisson distribution. The nature 
of this distribution is important because the Navy's policy is to acquire 
enough weapons (budgets permitting) to make the probability that the typical 
platform does not run out exceed some confidence level of approximately .9. 

The required inventory is quite sensitive to this confidence level when the 
distribution is geometric. 

The calculation of ordnance expenditures combines the distributions of 
the number of salvos expended to kill a particular number of targets and the 
number of targets encountered by a platform. The result is the distribution 
of the maximum number of salvos expended by any single platform. This 
distribution is then modified to reflect the existence of false targets, and 
finally an initial allowance for each combatant is calculated to satisfy the 
aforementioned confidence level. 

When the estimate of a platform's initial allowance is larger than its 
magazine capacity, replenishment is necessary. The replenishment model used 
here is a reservoir model. The reservoir draws down under heavy demand and 
refills to its limit during light demand. A deterministic refill size, the 
average refill, is used for replenishment. Replenishment demand for a group 
of platforms is obtained by aggregating individual demands while noting that 
the individual demands are correlated because of the fixed number of targets 
in the pool. Groups of platforms place demands on station ships which then 
place demands on depots. Failure to supply these demands increases the 
vulnerability of platforms and serves as a measure of effectiveness for the 
risk of a reduced inventory. The measure of effectiveness of the THREAT model 
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may be precisely stated as, at any selected level of confidence, (1) platforms 
receive enough ordnance, up to magazine capacities, to assure they do not run 
out; (2) given that platform requirements exceed their capacities, the 
logistics ships are loaded out, up to capacities, to assure that platforms do 
not run out; and (3) given that platform requirements exceed logistic ship 
capacities, the ashore depots have sufficient stocks to assure that platforms 
do not run out. The model assumes no delays and no ordnance losses in 
resupply . 

The foregoing discussion of requirements determination in the threat- 
oriented model makes no mention of attrition to friendly platforms. There is 
nonetheless a sense in which friendly attrition is built into the model; this 
may even be intuitively reasonable, since one effect of attrition ought to be 
to concentrate targets on certain platforms (the ones still surviving) as 
predicted by the model. Here is the sense in which attrition is "built-in": 
Imagine first that a certain number (a) of "attritions" are mixed in with the 
(k) targets. A platform unlucky enough to be chosen by an attrition is 
immediately sunk; unlike targets, attritions cannot be shot down. The 
sequence in which attritions and target kills appear is governed by a Markov 
chain with state (k,a). Conceptually, the model starts at event k+a and works 
back to event 1 in the representation 

(k,a) (k-l,a) ^ (k-l,a-l) ^ ... ^ (1,0) (0,0), 

where K denotes the occurrence of a kill and A denotes the occurrence of an 
attrition. The model assumes the following conditional probabilities at any 
stage; 

Pr((k-l,a) I (k,a)) = k / (k+ap) and 

Pr((k,a-1) I (k,a)) = ap / (k+ap). 
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This indicates that, for small p, target kills tend to occur late in the 
sequence and attritions early. For p = 1, any sequence of K's and A's is 
equally likely. A remarkable result (Reference [5], Appendix 0) is that as 
long as (a) is less than the number of platforms, the distribution of the 
number of kills per platform is independent of (a). In this sense then, 
attrition is built into the model. 

None of these attrition calculations is actually carried out in the 
threat-oriented model. The fact that the distribution of kills per platform 
is independent of (a) means that (a) might as well be set to 0; i.e., 
attrition can be ignored for purposes of calculating weapon allowances as long 
as the attrition mechanism described above is felt to be probabilistically 
realistic. There is no evidence that it is realistic. It is not implausible, 
however, and there is also no evidence to the contrary. 

2.3 Observations on NAVMOR 

The optimization part of the NNOR is incorporated within NAVMOR, more 
specifically in the part of NAVMOR where weapons compete on a cost/ 
effectiveness basis to be loaded on sorties. This is the only part of 
the NNOR where mathematical optimization can potentially affect the weapons 
mix, so it is worthwhile looking at it in some detail. We will do so in this 
section, staying alert to the possibility that the method in use may introduce 
biases into the weapon selection process. 

As this is written, NAVMOR is being run in support of POM 88, with the 
crucial input from PHASER being the number of sorties of various types that 
would be generated by all the aircraft the Navy expects to own in 1988 in the 



process of fighting a hypothetical war that begins then. PHASER would also 
provide similar inputs for wars beginning in later years, but the "outyears" 
are taken less seriously than 1988 in POM 88. These sorties are simply 
provided as gross totals; time is not represented explicitly in NAVMOR. There 
is no specific input list of targets, which is in keeping with the 
level-of-effort assumption that this is the part of warfare where weapon 
consumption is limited by the number of shooters, rather that the number of 
targets. The operating assumption is that the environment is "target rich", 
so that there is no possibility of running out of targets regardless of which 
weapons are carried. The object is not to kill all the targets (as it is in 
the threat-oriented case), but rather to kill targets cheaply. The general 
idea is to apply the input sorties as efficiently as possible, sum up the 
weapons required, and use the resulting total (augmented by peacetime 
consumption) as guidance in drafting the POM. 

"As efficiently as possible" means that weapons and tactics are chosen to 
maximize bang per buck as given by (1). The numerator k of (1) is described 
in [6] as "the JMEM determined expected kills (in n passes using tactic t to 
deliver i weapons of type w)". There are actually several ways to derive that 
quantity from information in the JMEM, since the JMEM tabulates only p(i) = 
"the probability of killing a target in one pass with i weapons" (let us agree 
to suppress dependence on w and t for the moment). To be definite, suppose 
that we are considering a sortie loaded with 12 weapons to be delivered in 3 
passes with 4 weapons in each pass. Let Q = l-XS(w,t) be the over-the-target 
attrition probability; this is the same expression that is involved in the 
denominator of (1). There are then at least four expressions that fit the 
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above English description of the numerator k: 

1) k = 3p(4) 

2) k = (l-p(4))^ 

3) k = p(4)(qV+Q^) 

4) k = p(4)(l+Q+Q^) 

The first expression corresponds to making three passes at three targets. 
The second corresponds to making three passes at one target. The third 
corresponds to making passes at three targets until attrition occurs, with the 
chance for attrition occurring before each pass. The fourth is the same as 
the third except that attrition occurs after each pass. There are other 
possible versions of 2) that include attrition before and after each pass. 

In a world where one expression is justified, use of a different one 
might lead to an unfortunate choice of weapons or tactics. The "powering-up" 
philosophy of 2), for example, is more likely than 1) (generalized to permit 
an arbitrary number of passes) to result in a single pass being optimal, and 
also more likely to result in selection of expensive weapons with high kill 
probabilities. Expression 4) is more likely than 3) to result in selection of 
standoff weapons for which Q is large. Expression 1) is more likely than 4) 
to result in the selection of aggressive tactics and weapons for which Q is 
small; there is a limit to this argument because Q also appears in the 
denominator as part of the sortie cost, but there is nonetheless a tendency in 
that direction. Further comparisons could be given, but we trust the point 
has been made; the English expression "average number of targets killed" is 
not definitive until details of combat have been fixed, and the manner of 
fixing those details affects the mathematical expression to be used, and in 
the process, the weapons that are ultimately selected. 
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Expression 1) is the one actually used in NAVMOR, this being deemed more 
appropriate than expression 2) because the environment is assumed to be target 
rich. Our vote is for the last expression. The fact that attrition is 
accounted for in the denominator does not excuse ignoring it in the numerator; 
attrition causes less targets to be killed as well as more aircraft to be 
lost. We choose 4) rather than 3) because the extra factor of Q can be 
thought of as a degradation factor for JMEM results, which do not deal 
explicitly with the effect of potential attrition on weapon delivery accuracy. 
However, 1), 3), and 4) are all essentially the same thing if Q is close to 1, 
as is typical of historical results. 

Of more concern is the attrition scaling factor X. It was mentioned in 
section 2.1 that PHASER provides an attrition goal to HAVMOR, and that MAVMOR 
adjusts X until computed attrition meets the goal. To see the problem with 
the procedure in use, assume that the tactical choice is essentially one of 
being aggressive (in which case the target kill probability and the attrition 
probability are both large) or conservative (in which case both of those 
quantities are relatively small). The actual tactical choice might be 
altitude or dive angle. Assume further that the initial value for x is such 
that aggregate attrition is too large. One would expect the system to react 
by using less aggressive tactics on the next pass, but in fact exactly the 
opposite will occur. Here is how it happens. Since attrition is too large, x 
is adjusted downwards. The effect of this is to increase Q on the next pass, 
so the optimization procedure will select more aggressive tactics. In spite 
of the more aggressive tactics, less attrition will be measured, and the 
constraint will indeed be met if the procedure is repeated several times. The 
difficulty is that the adjustments of tactics are intuitively in the wrong 
direction. This is easiest to see in the extreme case where X=0, in which 



case the system would report no attrition at all while simultaneously using 
whatever tactics maximize damage to the target in one pass (pure aggression). 

It might be argued that this "tactical instability" feature is not 
important because the NNOR is not used to determine tactics. However, the 
same issue exists for weapon selection, albeit not in such a pure form because 
weapons have a cost term in the denominator that does not vanish when X=0. 
Suppose, for example, that NAVMOR were to produce a mix of standoff weapons 
(think of them as conservative weapons) and iron bombs (aggressive weapons) in 
response to given inputs from PHASER, and that the inputs from PHASER were 
changed to reflect a goal of having less attrition. The NAVMOR output could 
very well shift toward iron bombs! NAVMOR responds to requests for less 
attrition by convincing itself that attrition is less likely in the real 
world, with the natural result that the tactics and weapons selected become 
more aggressive. 

The simplest way to fix the tactical instability feature would be to make 
attrition estimates that are absolute, rather than relative. The overall 
attrition could still be controlled to meet a constraint, if desired, by 
introducing a LaGrange multiplier on the cost of attrition. An increase in 
the multiplier would cause NAVMOR to produce less attrition by selecting more 
conservative tactics and weapons, rather than by changing its mind about the 
nature of the real world. This is easy enough to do mathematically; the 
difficult part of the fix is in getting absolute attrition estimates. 

A final observation about NAVMOR is that it is biased toward special 
purpose weapons because it effectively assumes that weather, for example, 
is known chronologically before weapons must be loaded onto sorties. NAVMOR 
shares this property with the Air Force's FAST ATTACK program. Since section 
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3.1 contains a detailed discussion of the issue, there is no need to go into 
it here. 

2.4 Observations on the THREAT Model 

The THREAT model separately generates the combat requirements for each 
weapon under consideration by partitioning the threat into sets of targets 
applicable only to individual weapons. This forecloses the capability of 
the model to make weapons tradeoff or substitutability decisions on any basis. 
Such decisions must be made at a higher level of aggregation than that treated 
within the model and, hence, are not made on the basis of actual, microlevel 
combat substitutability considerations. 

The threat partitioning, when combined with the "built-in" attrition 
aspect of the model, implies that targets serve no purpose other than to be 
destroyed. This is equivalent to the condition that attrition is independent 
of efforts to limit it. Even more extreme than NAVMOR, this means that 
different missions with the same weapon will result in probabilistically 
equivalent outcomes as long as only the target populations are the same. 
Related to this point, the model assumes that the only way that targets can 
avoid being destroyed, except when the maximum depth-of-fire has been reached, 
is by exhausting the ordnance of the platforms. This assumption ignores the 
effects of both leakers and saturation attacks on platform attrition. 

The model assumes ordnance losses occur only through attrition of plat- 
forms. The logistics pipeline is assumed to be attrition-free, so that 
neither station ships nor depots suffer attrition losses. Along with the 
assumption of instantaneous resupply at refill events, this suggests that 
further work should be directed toward logistics considerations in the model 
if the model is to remain in use. 
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2.5 Ordnance Programming Model 



The above models have provided no framework to compare alternative 
ordnance programs on the basis of costs and effectiveness. Because of this 
lack of an explicit methodology for programming of threat ordnance, the Center 
for Naval Analyses (CNA) was tasked to develop such a methodology. The 
resulting Ordnance Programming Model (0PM), see reference [7], provides a tool 
for trade-off analyses involving non-nuclear threat ordnance. 

The general methodology of the 0PM rests upon two points. The first is 
that the model must avoid the requirements , or threat-partitioning, approaches 
found in the NNOR in order that the substitutability of alternative weapons 
can be modeled. The second major point of the 0PM is that each run begins 
with a specific inventory of ordnance, and the model then calculates the cost 
of that specific inventory as well as several measures of effectiveness. This 
approach enables explicit trade-offs to be made among alternative weapon 
inventories and programs. 

Implementation of this methodology has resulted in a hierarchical model 
containing individual engagements which are aggregated up through missions, 
incidents, and theaters to define a scenario. The model is scenario-based and 
contains nine different predefined missions (AAW, ASW, and ASUW) plus 
user-defined missions as appropriate. There is an explicit handling of 
logistics and resupply considerations. The model is time-dependent and uses 
only expected values with accounting performed at the end of each period. The 
model has no wargaming capability and is run in a single pass through the 
scenario and its inputs. The following outcomes are available at the end of a 
scenario; number of targets killed, number of platforms surviving, quantity 
of ordnance fired, quantity of ordnance lost on attrited platforms, and 
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various user-defined measures of effectiveness which are functions of the 
previous outcomes (days of sustainability, survivability, etc.). 

The specific methodology of 0PM is implemented within engagements. Other 
levels of the model primarily provide accounting and input/ output function. An 
engagement consists of a single type of target using one of four alternative 
return fire assumptions. These assumptions are (1) no return fire, (2) 
targets return fire only after platforms fire, (3) platforms and targets fire 
simultaneously with explicit modeling of return fire characteristics, and (4) 
simultaneous fire with a simple exchange ratio. 

Engagements are assumed to be built up from Bernoulli trials and, hence, 
use the geometrical probability law in calculations. These calculations yield 
four outcome measures for each engagement: the number of targets killed, the 
number of platforms lost, the number of weapons expended, and the number of 
weapons lost on attrited platforms. 

The number of targets killed is defined as 

TARGETS KILLED = P *P *T*P, , 

da k ’ 

where 

P^ = Probability of detection = Pr( detection ) , 

P, = Pr(attack I detection), 

ot 

T = number of targets, and 

P|^ = Pr(kill I attack). 

The first three terms yield the number of targets attacked. 

The probability of kill given attack, Pj^ , is calculated from 

P, - 1 - (q, (1 - 
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where 



'’k ' ‘ - fk- 

P|^ = Pr(kill I single salvo), 

= Pr( reattack), and 

SA = salvos available per target attacked. 

This probability is built on the single-salvo and not the single-shot kill 

probability. This formulation also assumes that the platform fires the number 

of salvos which are available on the platform. 

The number of salvos available per target attacked, SA, is 

SA = (PE*PL) / ( P *P *T*(1-FA)*SS) , 

d a 

where 

PE = platforms engaged, 

PL = platform loadout, 

FA = false attack ratio, and 
SS = salvo size. 

In any given engagement, SA may have to be adjusted due to changes in ordnance 
remaining on the platform. If SA is more than one but less than the maximum 
number that can be fired considering the kinematics of the engagement, then 
the value computed above is used without adjustment. If the computed value is 
greater than the maximum, it is reduced to the maximum. If SA is less than 
one, it is set to one and the number of targets attacked is reduced in the 
above equation to yield an SA of one. The only other alternative when SA is 
less than one is to reduce the salvo size, and this was rejected on opera- 
tional grounds. 
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Total ordnance fired is calculated from 
TOTAL ORDNANCE FIRED = P *P *T*( 1-FA)*SS*SF , 

Q d 

where 

SF = expected salvos fired per target attacked. 

This formulation assumes that some targets which are fired upon are either 
false or outside of the correct firing envelope for the particular weapon. 
Expected salvos fired per target attacked, SF, is 

SF = (l-(q,^*p^)^^) / (l-(q,^*P^)). 

Total ordnance lost is computed 

TOTAL ORDNANCE LOST = (TOTAL PLATFORMS L0ST)*(PL)*( (l-RR)/2) 

where 

RR = platform reserve rate. 

The number of platforms lost can be calculated using any of the four 
assumptions on return fire. This formulation assumes that platforms have an 
"average" magazine load which is midway between loadout and their reserves just 
before loadout. The amount of loadout depends upon the amount of ordnance of 
that type within the theater. 

On a given platform, substitution can occur within weapons of the same 
general type since longer range weapons are assumed to be used up first and 
shorter range weapons substituted during a given mission. Note that platform 
attrition is modeled as an integral part of each engagement. 

Resupply is addressed by the 0PM through first deducting the ordnance in 
the maintenance pipeline. Rear echelon stocks are available across theaters, 
but there is no attrition of the rear echelon stocks. These stocks are 
attrited during inter-theater (once) and intra-theater (twice) moves. 

Attrition in resupply can occur to the ordnance but not to the resupply ships. 
Platforms can be replenished only between missions, with submarines handled as 
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a special case. 0PM does not address the scheduling of ordnance since within 
theater ordnance is available instantaneously. 

Criticisms of the 0PM revolve around some of its resupply assumptions and 
the validity of some of its calculations in certain circumstances. Integration 
of more realistic assumptions into several components of the model could include 
the addition of intra-theater scheduling and attrition of resupply ships. 
Although the model is claimed to be an expected value model, it is not clear 
that some of the calculations detailed above can be carried out for nonintegral 
values of some of the parameters. 

It should be noted, however, that the 0PM is the only model we encountered 
which attempts to directly answer our first question posed above concerning the 
cost-effectiveness of alternative weapons mixes. Although imperfect, it offers 
the capability to make cost-effective weapon trade-off decisions. 

3. Current Air Force Methods (Sabre Mix) 

Like the Navy, the Air Force has distinct models for air-to-air and air- 
to-ground munitions. Only the air-to-ground methods will be reviewed here. 

The Air Force's method for programming air-to-ground munitions is the Sabre 
Mix collection of models. The methodology of Sabre Mix is very similar to the 
Navy's NAVMOR. The keystone of Sabre Mix is HEAVY ATTACK (more recently known 
as FAST ATTACK), which is a close logical descendant of a model constructed by 
Clasen, et al [2]. The original model assigned aircraft sorties to targets 
"optimally" in a single period war, with the crucial inputs being 




and with the crucial outputs being 



s.. = number of sorties of type i assigned to targets of type j. 
^ J 



The rest of the models in Sabre Mix are designed to make HEAVY ATTACK usable 
in a multi-period war where p. • depends on weather and weapons carried 

• \j 

and where the desired output includes information about munition consumption. 

Formally, if one is given the numbers 

p. .. = average number of type j targets killed by a type i 

sortie loaded with weapon type k when the weather is w, 

the problem is to somehow "process" all those numbers to obtain the p^.^ inputs 

to HEAVY ATTACK, as well as to recover weapon usage from the output. The 

processing is performed by WEAPONEER (which produces the ^nd SELECTOR 

(which turns p. .. into p.. and also provides attrition estimates). These 

1 J KW 1 J 

three models constitute Sabre Mix. The passage of time is handled by simply 
running HEAVY ATTACK once in each time period. See Lord [3] for further 
details. 

Subsequent sections will discuss specific aspects of this process that 

lead to bias or inaccuracy. Some of these problems are due to ultimate 

reliance on a model (HEAVY ATTACK) that seemingly does not have enough 

decision variables. Why not simply introduce the variables 

s... . = number of sorties of type i loaded with weapon type k 

^ assigned to targets of type j in weather type w in time 
period t 

and solve one big optimization problem, rather than trying to use the 
aggregated, myopic Sabre Mix process? The idea has been proposed and even 
tried. Hartman [1] gives a concise description of the TAC RESOURCER model, 
which does exactly that, and Lord [3] decribes how HEAVY ATTACK could be 
modified along those lines. The Air Force continues to rely on Sabre Mix, 
however. One major reason for this is no doubt solution time, since, of 
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course, models with more variables take longer to solve. Improved techniques 
and hardware will eventually mitigate this problem. Other reasons for this 
continued reliance are the expanded data base required for a more detailed 
model and a desire for year-to-year consistency. 

The following sections deal with a specific parts of the Sabre Mix 
methodology. We will make no attempt to deal with grand questions such as "Can 
any approach based on a priori assessment of target values succeed?", "Why is 
it that the enemy has no choices here?", or "Why is least cost per kill the 
proper criterion for choosing weapon loads?" These questions have been raised in 
other places (Defense Science Board [12] and Bracken, et al [8]). Instead, we 
will carefully examine some of the inner workings of the Sabre Mix methodology. 
3.1 Randomness and Optimization 

Warfare is essentially unpredictable on account of the enemy's motivation to 
make it so. That can't be helped without employing Game Theory, which would 
greatly complicate the model and the optimization problem. Unintelligent sources 
of randomness can be and are taken into account in Sabre Mix, however. The two 
most important sources are: 

a) the unpredictability of the results of a given sortie, 
even knowing the weather, and 

b) the weather. 

The first source of randomness is handled by ignoring it, which is 

reasonable because individual sources of randomness have a tendency to 

average themselves out when computing the objective function "total target 

value killed." Since p. • is itself an average, the same thing is 

^ J 

true of the objective function. 



L 
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The second source of randomness is handled in SELECTOR by calculating 

Plj ‘ I Pw Pfjkw’ 

where is the probability that the weather will be in one of the six 
possible "bands". This assumption tends to have the weapon type always 
optimized for the- weather (the actual optimization is on cost per kill, 
rather than on directly, but that feature is not important for the 

moment). An alternative would be to compute 

p . . ' = max y p p . , 

1 ^ ^^w^ijkw’ 

which would correspond to the assumption that the true nature of the weather 
will be discovered after the choice of weapon type is made. The current (p^-J 

method will load specialized weapons compared to the alternative, v/hich will 
tend to select weapons that are robust to weather type. If this alternative 
assumption is true for a significant portion of the time in actual combat, the 
current method is biased to that extent toward specialized weapons. 

Just as Sabre Mix assumes that weather is known before weapons are selected, 
it also assumes the weather is known only after aircraft are assigned to targets; 
this is manifest because the inputs p. . to HEAVY ATTACK are not subscripted for 
weather. Sabre Mix must therefore be suspected of having a bias toward robust 
aircraft (actually robust ai rcraft- target assignments), since fair weather 
aircraft are in effect forced to fight in foul weather conditions with the prooer 
climatological frequencies. This would be more worrisome if Sabre Mix were used 
to recommend aircraft inventories, since all-weather aircraft are over-valued by 
the procedure. It is not clear what effect this feature has on weapon selection, 
but there very well could be one. 
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The point is that sometimes aircraft can be assigned to targets consider- 
ing the weather, and that sometimes weapons must be assigned to aircraft with- 
out knowing weather (or knowing it incorrectly ) . Sabre Mix ignores both of these 
possibilities, and is therefore biased in favor of robust aircraft and against 
robust weapons. There is no simple cure for this problem, although the magnitude 
of the weapon bias could at least be tested bv using p..’ (instead of P..) in 
HEAVY ATTACK and comparing results. Lord [3] also proposed running HEAVY ATTACK 
six times, once for each weather band. This would be a test of the aircraft 
bias. 

3 . 2 HEAVY ATTACK objecti ve func ti on 

The objective function in HEAVY ATTACK contains a "tuning parameter" Cj 
for each target type that has something to do with the extent to which dead 
targets can be distinguished from live ones. The parameter is not physically 
motivated, and the obvious way of motivating it would lead to the use of an 
objective function that differs from the one currently in use. The purpose of 
this section is to derive this new objective function and comment on the 
significance of the issue. 

The problem can be illustrated with a one-period war where there are T 
identical targets that are gradually killed by sorties flown throughout the 
period. A fraction, 1-C, of the taroets that are killed are assessed as such. 
This statement is the physical motivation mentioned earlier, since it is 
testable. The rest (C) serve to dilute the efforts of the aircraft in causing 
subsequent attrition, since they appear to remain alive and will continue to do 
so even if re-attacked. t If the number of dead targets is Y, the probability 
that any surviving target is chosen from the T-Y+CY that appear to be surviving 



t A similar analysis can be carried out if the re-attack process may reveal 
apparently alive targets to be actually dead. The HEAVY ATTACK formula 
corresponds to neither assumption. 
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is the ratio (T-Y)/(T-Y+CY) . Now let X be the number of targets that would be 
killed if there were no dilution; X is essentially a scaled number of sorties, 
and is given in HEAVY ATTACK by P.. S... The rate at which Y changes 
per unit increase in X is just the probability that a live target is attacked, 
which in turn depends on Y: 

dY T-Y 
dX " T-Y+CY 

an ordinary differential equation that can be solved by separating variables. 
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The solution is 



X/T = (1-C)(Y/T)- C ln(l-Y/T) 

The formula relating Y to X in HEAVY ATTACK is Y/T = (1- exp (-CX/T))/C, which 
is not the same thing. The two formulas agree when C = 1 (in which case both 
are Y/T = (1-exp (-X/T)) or when C = 0 (in which case both are Y = X after 
L' Hopital's rule is applied to the HEAVY ATTACK formula). In general, however, 
there are differences, as can be seen in Figure 2, where the HEAVY ATTACK (HA) 
formula is compared with the new (MRS) formula: 



MODEL COMPARISONS 



o 




Figure 2 
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Figure Z makes it clear that HA differs significantly from NPS, requiring 
less effort for a given fraction killed. The difference, while significant, may 
not be important. An argument for unimportance is that in practice C is 
generally set to one of the extremes 0 or 1, which are exactly the two cases 
where the formulas agree. A counter-argument is that nobody has ever before 
been sure what C represented except in those two cases, so that the prevalence of 
0 and 1 in inputs does not indicate anything about actual battle conditions. 
Another argument for unimportance is that allocation of given assets is probably 
insensitive to the objective function anyway. The counterargument is that Sabre 
Mix can also be used to investigate the size of the required asset base, for 
which purpose the objective function is more important. 

In summary, the Sabre Mix objective function currently involves a parameter 
C that has never been described physically. If the meaning of C is taken to be 
"fraction of killed targets that function as decoys", the NPS formula results, 
rather than the HA formula. Replacing HA with NPS would be significant in some 
cases. 

4 . Brief Review of Other Proposed Method s 
4.1 Speci al Versus General -Purpose Weapons 

Mangel and Nickel [11] express concern that current methods are biased in 
favor of special purpose weapons, the difficulty being that current methods do 
not give general purpose weapons sufficient credit for their greater flexibility 
in the face of uncertainty about target type, number, and order of arrival (see 
section 3.1). They propose a two-stage optimization procedure for properly 
handling these uncertainty issues. In the first stage, one calculates the best 
employment of a variety of v/eapon mixes against a variety of target sets, 
together with the resultant payoff. In the second stage, the weapon mix is 
selected to maximize the expected payoff, utilizing the assumed known probability 
distribution over target sets. 
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Symbolically, if V({N.1|T) is the payoff when weapon mix (fl.l is applied 
optimally to target set T, then it is E^(V( |T)) that should be optimized, 

where E-j- represents expectation with respect to target set. Current procedures 

optimize V( {N^.} |Ej(T)) , which is simpler computationally but essentially the 
wrong thing to do. 

Optimizing V( {N^. } |E.j.(T) ) is an example of "expected-value-analysis". One 
simply proceeds as if a random quantity (T, in this case) could be replaced by 
its mean without affecting the conclusions of the analysis. The procedure is 
faulty, but has two things to recommend it: 

a) One does not really "replace" a random variable by its mean. The 
variable involved is never thought of as being random at all. This at 
has the virtue of minimizing requirements for data collection, since no 
probability distribution is required. 

b) Computational requirements are minimal, since only one value of the 
quantity need be considered. 

The question is whether the gains in the quality of the answer are worth the 
costs of data collection and computation. For the weapon selection (stage 2) and 
allocation (stage 1) problem that they consider. Mangel and Nickel present 
evidence that expected-value-analysis is costly in terms of bias in the weapon 
mix ultimately selected. They also discuss some methods for minimizing the 
computational requirements of the "correct" method, amongst them being the use of 
heuristics in stage 1 and of stochastic approximation in stage 2. For problems 
on the scale of those considered by MAVMDR or Sabre Mix, however, the 
computational burden would still (we think) be prohibitive. 

The issue of special versus general purpose weapons is also considered in 
Johnson and Loane [13]. 
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4.2 RAF Model 



Daniel, et al [14] report a conventional weapon procurement model that deals 
with the flexibility issue without introducing any probability distributions. 
Their method is Linear Programming (LP). The LP that they use is reproduced 
below: 

objective function 

Maximize ar - d. + 6V (E,-i,u.. ) (a, y and 6 weights, y > a > 6) 
k ik 

Subject to: 

Production capacity 

n^. <A. V-i 

Budget limit 

y C.n.<Q 
1 1 



Storage constraint 



y S.h.<B 
1 1 

1 



Flexibility constraint 



■■ - -Ik- « 



Basic sorties constraint 



^“ik* ^k^k = ^k « 



Flexible sorties constraint 



Weapon pool constraint 



Flexible weapon constraint 



Total weapon stock 




< V *k 

k*k„ 





w^. > 0 -V-i 




k.k^ 



h. - n. - L. AAi 
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Variables 



d|^ sortie shortfall in role k (per planned sortie). If insufficient aircraft 
loads of weapons are available to meet the planned sortie allocation to 
roles, then d|^T|^ is the shortfall in sorties in role k. is defined 
below. (Note that one aircraft load of weapons is needed for one sortie.) 

h^. total number of aircraft loads of weapon type i held in stockpile. 

n^ number of aircraft loads of weapon type i purchased. 

r flexibility of the solution. If the maximum possible number of resources 

are concentrated on a given role, this gives a particular increase in 
effectiveness in that role above the planned level. The minimum value of 
this, over all roles, is the value of r. 

u^.|^ number of aircraft loads of weapon type i used to satisfy the planned 
sortie allocation in role k. 

v^.|^ number of aircraft loads of weapon type i (in excess of those to satisfy 
the planned sortie allocation) allocated to role k. 

w^. number of aircraft loads of weapon type i held in a pool for flexibility. 

Coefficients and constants 

A^. production capacity of weapon type i (in aircraft loads). 

B total storage volume available. 

C^. cost of an aircraft load of weapon type i. 

E^.|^ effectiveness of weapon type i in role k. 

L^. initial stock level of weapon type i (in aircraft loads). 

P|^ proportion of planned sorties in role k which can be switched from this 

role. 

Q total budget available. 

storage volume occupied by one aircraft load of weapon type i. 

T|^ planned number of sorties allocated to role k. 
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Note that the objective function is essentially heirarchical . The shortfalls d, 

k 

will be forced to 0 if possible because y Is so large, next r will be made as large 
as possible consistent with the being 0, etc. The basic idea is that sufficient 
weapons must be bought to supply the planned sorties, with subsequent deviations 
from the plan being handled from a pool of flexible weapons. The flexible weapons 
have two sources: those purchased for the purpose (w^. ) and those obtained by 
transferring weapons from other roles. Note that if the transfer fractions 
were variables, rather than data, the optimal values would all be whatever maximum 
is permitted. Note also that aircraft do not enter explicitly, serving merely as 
the units in which weapons are measured. 

The authors report that the model behaves sensibly in several respects. As the 
budget level Q is reduced, for example, the effect is to replace expensive, 
effective but specialized weapons with cheaper, more versatile but less effective 
weapons, while the total weapon stock remains constant over a broad range of Q. 

The model appears to have been well received. The authors state that "This 
approach appears to have been successful in answering immediate Air Staff problems 
and can certainly be claimed to have influenced decisions. It is, of course, too 
soon to assess whether the method, or a derivative, will be used more widely." 
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